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Abstract —We consider a three-dimensional problem of steering 
a nonholonomic vehicle to seek an unknown source of a spatially 
distributed signal field without any position measurement. In 
the literature, there exists an extremum seeking-based strategy 
under a constant forward velocity and tunable pitch and yaw 
velocities. Obviously, the vehicle with a constant forward velocity 
may exhibit certain overshoots in the seeking process and can 
not slow down even it approaches the source. To resolve this 
undesired behavior, this paper proposes a regulation strategy for 
the forward velocity along with the pitch and yaw velocities. 
Under such a strategy, the vehicle slows down near the source 
and stays within a small area as if it comes to a full stop, 
and controllers for angular velocities become succinct. We prove 
the local exponential convergence via the averaging technique. 
Finally, the theoretical results are illustrated with simulations. 

Index Terms —Extremum seeking, adaptive control, nonholo¬ 
nomic vehicle, averaging. 

I. Introduction 

R ecently there has been a growing interest in the 
study of steering single or multiple autonomous agents 
to seek the source of a scalar signal field. The signal could be 
thermal, electromagnetic, acoustic, or the concentration of a 
chemical or biological agent. The strength of the signal field 
is usually assumed to decay away from the source, and the 
agent only has the capability of measuring the signal strength 
at its present location. However, the information about the 
spatial distribution of the signal field is unavailable. Such 
a source seeking problem is of interest in a wide range of 
applications, including explosive detection, drug detection, 
localizing the sources of hazardous chemicals leakage or 
pollutants, localizing hydrothermal vents, etc. 

Various methods have been proposed to study source seek¬ 
ing problems and related issues. Porat and Nehorai HI ex¬ 
plored the use of moving sensors for detecting and localizing 
vapor-emitting sources by computing the gradient of the 
Cramer-Rao bound on the location error. Ogren et al. d 
addressed the gradient climbing question of a mobile sensor 
network. Pang and Earrell d provided a source-likelihood 
mapping approach based on Bayesian inference methods to 
estimate a likelihood map for the location of the source of 
a chemical plume. Demetriou et al. d proposed a coupled 
controls-computational fluids approach for the estimation of 
the signal field in a two-dimensional (2-D) domain with a 
Lyapunov-guided sensing aerial vehicle. 
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Different from m-a where the agents are also assumed 
to be capable of sensing their positions, this paper considers 
the three-dimensional (3-D) problem of steering a single 
nonholonomic vehicle to seek the source without the vehi¬ 
cle’s position information. This consideration is motivated 
by vehicles operated in environments where their position 
information is unavailable or costly, such as urban, under¬ 
ground and underwater environments. The lack of position 
information renders the guidance of the vehicle interesting 
and challenging, as most of traditional searching strategies 
fail to work under this framework. In addition, we use only 
one vehicle in the mission in contrast with HI, El- While 
the collaboration among multiple vehicles may improve the 
search efficiency due to the capacity in exploring several 
locations simultaneously and exchanging data in real time, 
a single vehicle can collect field values more freely with 
lower hardware overhead. As in Q-m, this paper focuses 
on seeking a source whose signal strength depends only on 
the distance between the vehicle and the source, . 

Extremum seeking is a real-time model free gradient es¬ 
timation method to optimize the steady-state output of a 
nonlinear system, without any explicit knowledge about the 
input-output map other than the existence of an extremum 
M- It has been implemented successfully in many different 
areas m-ca. Specially, a series of studies suggest that it 
is an effective approach for source seeking problems with¬ 
out position information Ha-Ha. Employing the extremum 
seeking method, two distinct strategies for source seeking 
problems in a plane were designed. While Zhang et al. 0 
considered controlling the vehicle by keeping the angular 
velocity constant and tuning the forward velocity, Cochran and 
Krstic 0 considered controlling the vehicle by keeping the 
forward velocity constant and tuning the angular velocity. Liu 
and Krstic 0, 0 replaced the periodic sinusoidal excitation 
with the stochastic excitation and redesigned the extremum 
seeking method. Ghods and Krstic ITOl regulated both forward 
velocity and angular velocity with the intent of bringing the 
vehicle to a stop near the source. 

While most of the previous research focused on source 
seeking problems in a plane, there are few work on the three 
dimension case 0, EqI. The 3-D model is more complicated 
due to more optional vehicle movements, thus the methods 
in the 2-D workspace can not be directly applied to the 3-D 
workspace. However, in many engineering applications source 
seeking in the 3-D workspace attracts more concern. Two 
control schemes in the 3-D workspace were presented by 
Cochran et al. 0. Both schemes considered a vehicle with 
a constant forward velocity. Whereas it is similar to the 2- 
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D case, a constant forward velocity results in complicated 
asymptotic behaviors of the vehicle. As a result, the vehicle 
cannot settle when it approaches close to the source. Instead 
it exhibits certain overshoots and finally revolves around the 
source with a relatively large forward velocity. Note that a 
small constant forward velocity may improve the asymptotic 
performance, but the convergence rate decreases. Matveev et 
al. II 20 I proposed a hybrid controller without estimation of the 
field gradient, which is justified to be of global convergence 
to the source of a generic field. However, they employed a 
vehicle with a constant forward velocity as well, which again 
results in the undesired performance similar to 0. 

To overcome the shortcoming induced by a constant forward 
velocity in the 2-D case, an approach that regulates both 
forward velocity and angular velocity was proposed by Ghods 
and Krstic Qo). In this paper the approach is extended to 3-D 
workspace, aiming at bringing the vehicle to a stop around 
the source. While only one angular velocity is considered in 
the 2-D case, we must address two angular velocities (pitch 
and yaw velocities) simultaneously in the 3-D case. The two 
angular velocities and the forward velocity interwine with each 
other, which makes the design and analysis of the control 
scheme challenging. A preliminary version of this work has 
been reported in WCICA 2014 1(211 . However, we establish a 
rigorous proof of the stability and provide a rule on parameter 
selection for the controllers in this paper. 

We tune both angular velocities and forward velocity simul¬ 
taneously in contrast with 0, 1201 . Under a tunable forward 
velocity we can slow down the vehicle around the source to 
get closer to the source without decreasing the convergence 
rate. The forward velocity is controlled to be small near the 
source, which eliminates the undesired overshoots. Moreover, 
the controllers for angular velocities can be simplified by 
removing extra items for eliminating the overshoots caused by 
a constant large forward velocity. We obtain two phenomena 
for the static source that produces a signal field with spherical 
level sets. In one phenomenon the vehicle eventually stays 
within a small area around the source as if it “stops” as 
expected. In the other it revolves in an annular attractor around 
the source, which is similar to 0 but with simpler controllers. 

The rest of the paper is organized as follows. In Section 2 we 
describe the 3-D nonholonomic source seeking problem and 
present our extremum seeking scheme. In Section 3 we derive 
an averaged system and prove local exponential convergence 
for the static source that produces a signal field with spherical 
level sets. In Section 4 we provide a rule on the parameter 
selection for the control scheme. In Section 5 we include 
simulation results to illustrate the behaviors of the vehicle 
under different scenarios. 


H. Problem Description and Control Scheme 

In this section we firstly describe the 3-D nonholonomic 
vehicle model for the task of the source seeking mission. 
Then we present our extremum seeking scheme and provide 
an intuitive interpretation of the control scheme. 


z 



Fig. 1. Geometric interpretation of vehicle model. 


A. Problem Description 

As in 0, we consider an autonomous vehicle under non¬ 
holonomic constraints, see Fig. [T] for an illustration. The 
vehicle is equipped with actuators which are used to impart 
forward, pitch and yaw velocities. The diagram in Fig. [T] 
depicts the position, heading, forward, pitch and yaw velocities 
for the vehicle center and sensor. The sensor Vg is mounted at 
a distance R away from the vehicle center r^. The azimuthal 
angle a defines the pitch, whose velocity is governed by 
The polar angle 9 defines the yaw, whose velocity is governed 
by ijjg. The forward velocity, or surge velocity, is defined by 
V. Note that the sensor is mounted at the tip of the vehicle, 
thus the roll velocity and angle do not affect the measurement 
and movement of the vehicle and can be neglected. 

The kinematic equations of motion for the vehicle center 
are 



cos(a) cos{9) 


V 

cos(a) sin(0) 

(1) 


sin(Q;) 


V'c 

(2) 


(3) 


where Tc = {xciJc^ Zc)- The sensor is located at 


Ts =rc + R 


cos(a) cos(0) 
cos(a) sin(0) 
sin(a) 


(4) 


The task of vehicle is to seek a source that emits a spatially 
distributed signal. The signal strength at the location r is 
denoted by J = f (r), which has an isolated local maximum 
/* = /(t*) where r* denotes the source location, or the local 
maximizer of /. Here the isolated maximum means that there 
is only one source in the searching area. The signal strength 
J decays away from the source r*, but other information 
about J, such as the shape of / and the position of r*, is 
unknown. Furthermore, the information of the vehicle’s posi¬ 
tion is unavailable, which makes traditional gradient searching 
strategy fail to work. Our objective of this work is to design a 
control scheme to steer the nonholonomic vehicle to the source 
without any vehicle’s position information. 
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Fig. 2. Block diagram of source seeking via tuning of forward, pitch and 
yaw velocities of the vehicle. 



Fig. 3. Block diagram of source seeking in (6). 


B. Control Scheme 

To achieve the goal of seeking the source, we employ an 
extremum seeking method to tune the pitch and yaw velocities 
{'tpa and ipg) directly and the forward velocity (u) indirectly, 
which is in contrast with in as they assume the forward 
velocity to be constant. It is worth mentioning that the 2- 
D case has been studied in ifTOl . However, the 3-D case 
is more intricate and challenging and remains outstanding. 
Firstly, the control scheme in Col for the 2-D case is no 
longer applicable. We consider more angular variables while 
the utilizable information is still constrained to the signal 
strength. In fact two extra states are added to analyze the 
dynamics of the system for the extra dimension. Secondly, 
the two angular velocities and the forward velocity interact 
with each other by affecting the motion and movement of 
the vehicle. The coupling of the three velocities obviously 
complicates the stability analysis of the vehicle dynamics and 
involves new challenges. Thus, we need to redesign the control 
scheme. 

Our control scheme is depicted by the block diagram in 


Fig. m The control laws are given by 

V = V, + bt (5) 

ijja = aw cos(a;f) + Ca^sin(a;f), (6) 

Tpe = —ow sin(a;f) + cos(wt), (7) 


where w is the probing frequency, the parameters a, Cq, eg, 
b and 14 are positive and will affect the performance of the 
approach, J is the sensor reading, and ^ is the output of the 
washout filter In the sequel, we shall provide a rule 

for designing the above parameters. 

In our control scheme in Fig. |2] the pitch velocity ipa and 
the yaw velocity ipg are tuned according to the idea of the 
basic extremum seeking tuning law Il22l . The perturbation 
terms, aa;cos(wf) and —ow sin(a;f), are added to persistently 
excite the system while the corresponding demodulation terms, 
5 sin(a;f) and ^ cos(w<), are used to estimate the gradient of the 


nonlinear map /. The washout filter is used to eliminate 
the “DC component” of the sensor reading J. The forward 
velocity v is designed to be positively correlated to the output 
^ of the washout filter, since ^ describes the variation of the 
sensor reading J in some sense. As a result, the vehicle would 
speed up when heading towards the source, and slow down 
when deviating from the source. A detailed stability analysis 
of this scheme for the static source will be shown in next 
section. 

It is worthy mentioning that our control scheme is quite 
different from the one in 0, which is given in Fig. [3 In their 
work a vehicle with a constant forward velocity is considered. 
Employing the basic extremum seeking method, the vehicle 
cannot settle even if it has reached the source due to the use of 
a constant forward velocity. In addition, it easily overshoots the 
source and turns around. The process may repeat for a while 
before the vehicle finally revolves around the source with a 
constant velocity. In |6l a so-called “d-item” is added to tune 
the angular velocities, aiming at improving the performance. 
Nevertheless, a constant forward velocity leads to complicated 
asymptotic behaviors of the vehicle. 

Instead of using a constant forward velocity, we tune the 
forward velocity along with the pitch and yaw velocities. 
This intuitively is a better way to control the vehicle, as 
we are able to smartly adjust the vehicle to speed up, slow 
down, or stop depending on different circumstances. Besides, 
we can conveniently maneuvre the angular velocities under 
a tunable forward velocity, which is evident from Fig. |2] 
While the control scheme in 0 employed relatively complex 
controllers as shown in Fig. [3 we can simply employ the 
basic extremum seeking controllers. This succinct feature 
substantially simplifies the parameter design of the control 
scheme. 

III. Stability of the Closed-loop System 

It is obvious that the dynamics of the closed-loop system 
is complicated due to the nonlinearities of the vehicle model 
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and the signal map and the time varying forcing applied by 
extremum seeking. Additionally, more coupled state variables 
are involved to analyze the dynamics than in the 2-D case. In 
this section we focus on a static source that produces a signal 
field with spherical level sets and employ an averaging method 
1^ to analyze the stability of the system. 

In this work, the distribution of the signal field is assumed 
to depend only on the distance from the sensor to the source. 
Since we are concerned with the local convergence, it is 
sensible to apply the Taylor series expansion around the source 
and omit higher order terms, which yields a quadratic map. 
Thus, the quadratic map is an approximation of the distribution 
of the signal field in a local sense, and takes the following form 

J = f{rs)=r-qr\rs-r*\\ (9) 


where r* is the unknown maximizer and denotes the location 
of source, /* = /(r*) is the unknown maximum and is an 
unknown positive constant. 

To analyze the stability of the closed-loop system, we define 
an output error variable 

e=^[J]-r, (10) 

which allows us to express the output of the washout filter as 


Moreover, we can easily obtain that e = h^. By inserting 
the control law (|5]l-(l8]l into the system ([T]i-(|4li, the closed-loop 
system is written as 


r-c 

a 

0 

e 


Ts 


{Vc + bO 
aw cos(a;f) 


cos(a) cos(0) 
cos(a) sin(0) , 
sin(a) 

- Ca^ sin(wf), 


—aw sin(a;f) -f cg^ cos{wt), 


h^, 

-qr\rs 
Tc + i? 


cos(a) cos(0) 
cos(a) sin(0) 
sin(a) 


( 11 ) 

( 12 ) 

(13) 

(14) 

(15) 

(16) 


The closed-loop system (fTTTi - ifThl l involves six variables, and 
it is difficult to analyze the system directly. We re-express the 
system by variable transformation and time scale change for 
further discussion firstly. Noting the signal strength J depends 
only on the distance between the vehicle and the source, we 
redefine the position of the vehicle center in its spherical 
coordinates form. Finally we successfully reduce the system 
order by defining an error variable. To this end, define shifted 
variables by 


fc = Tc-r* , 
a = a —asin(w<), 
9 = 9 — acos{wt), 

e = e + qrR^^ 


and introduce a time scale change 


T = Wt. 


The dynamics of the shifted system is given by 

dfc Vc + 


dr w 

da 


cos(a -I- a sin(r)) cos{9 + a cos(t)) 
cos(d -I- asin(r)) sin(0 + acos(r)) 
sin(Q: + a sin(T)) 


sin(r), 

dO cg^ 

— = -cos(r), 

dr w 
de 

dr w 


For further analysis we now redefine fc by its spherical 
coordinates 


rc = 


-Vc = Tc 


tan(a*) 

tan(d*) 


cos(a*) cos(0*) 
cos(q;*) sin(d*) 
sin(a*) 

Zc 


\/dl + yl ’ 


dc 

Xc 


where fc is the distance between the vehicle center and the 
source, a* and 6* represent the azimuthal angle and polar 
angle towards the source when the vehicle is at fc respectively. 
By using these new definitions, the expression of ^ is 

C = -qr{fl + - 2fcR^c) - e, 

= cos(d -f asin(r)) cos(a*) cos(d — 9 * -\- acos(T)) 

+ sin(d -f a sin(T)) sin(a*), 

and the resulting dynamics is 


“'c _ dr ^ dr dr 

dr fc 

iVc + bO^c 


da* _ + - ^fV^c + Vc 

dr 


Vc + b^ 


sin(d -I- asin(r)) cos(a*)— 


cos(d -I- asin(r)) sin(a*) cos(d — 9* + acos(T)) 

j/1* dye dxc 

d9 ^ ~ y<=l[7 

dr yl + xl 

Vc + b^ 


wfc cos(a*) 
cos(d -I- a sin(T)) sin(0 — 9* + a cos(r)) . 


The system order can be reduced from six to five by defining 
an error variable 0 = 0 — 9*, resulting in the following error 
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system 

dfc _ {Vc + b^)^c 

dr 


OJ 

da* Vc + b^ 
dr 


Lore 


sin(Q! + asin(r)) cos(a*) 


— cos(d + asin(r)) sin(a*) cos(0 + acos(r)) 

da Ca^ . . , 

— =-sin t), 

CLT UJ 

dO ce^ ^ ^ , Vc + b^ 

CLT OJ ojrccos{a*) 

cos(d + asin(r)) sin(0 + acos(r)) 


de h 
dr UJ ’ 


(17) 

(18) 

(19) 

( 20 ) 
( 21 ) 


where 




dr 


2bqrRrc 


UJ 


c 

>c 


UJ 


{hqr{frr + - V,) ‘^hrR 

— -^ave-^ 


dr 

da^^^ 

dr 

dQ^ve 

dr 


UJT^ 


ave sin 


UJ 


2caqrRr^ 


2c0qrRr^'^^ 

^ ^ ^ c t ave 

sc 
UJ 


2bqrR 


UJ cos(a*®'™) 


COS sin 
ave 


(Vc - 6 gr(rr®) - cossin 

^ -I \ C / ^ 


l^fave cos(q;*'^™) 


d^ ^ _ h^a 

j V c ) ^ 

dr UJ UJ 


2hqrR -avefave 
^ c Sc 1 


UJ 


c2ave _ 

Sc 


cos2(a*=^'^®) 


Jo(2-\/2a) cos(2a’^''®) cos(2d®'™) 


+Jo(2a) 008(20®"™) + cos(20®'™) 

+ - - [1 - ^o(2a) cos(2d®™)] 

+ sin(2a*®™) sin(2a®™) cos(r™), 


^ave = Jo(a)cos(a*®™)sin(d®™) 

- JoU/2a) sin(a*®™) cos(d®™) cos(r™), 
Jo(A/5a) 


C ave _ 

Sc — 


■ cos(2a*®™) sin(2d®™) cos(6»®™) 


sin(2a*®™) 


Jo(2-\/2a) cos(2d®™) cos(20®™) 


C ave _ 

SC — 


+3Jo(2o) cos(2d®™) + Jo(2a) cos(2r™) - 1 

Ji{V2a) 


V2 


■ cos(a*®™) sin(d®™) cos(6»®™) 


+ Ji(a) sin(a*®™)cos(d®™), 

Ji(y2a) 


^ = -qrfc + 2qrRfc^c - e, 

= cos(d + asin(T)) cos(a*) cos(0 + acos(r)) 

+ sin(d + asin(T)) sin(a*). 

Note that the system equations are periodic in 27r. According 
to the averaging method in ll23l . the solution of the error 
system can be well approximated by the solution of the 
corresponding “average system”. Averaging the error system 
in a period 27r, we obtain an average error system 


C ave _ 

coasin _ Jo( 2 y/ 2 a) 

- i 

do(2a) 

4 

JoiVba 


cos(a*®™) cos(d®™) sin(6»®™), 
cos(a*®™) cos(2d®™) sin(2r™) 


cos(a*®™) sin(2r™) 
sin(a*®™) sin(2d®™) sin(d®™), 


^ COS sin 
- ave 


= MV2a) cos(d®™)sin(d®™). 


By setting the right hand side of (l22]) -(l26ll to be zero, we 
obtain the following four equilibria with details being provided 
in Appendix 1^1 


( 22 ) 


eql eq 1 eql ~ eql eql 

~ave ^ *ave ^ ^ave ^ ^ave ^ 

= [71,0,0,0,61], 

-ave®*^^ „ -ave^'^^ ^ave®'^^ 


= [-7i,0,0,7r,ei], 

-ave'i^* „*ave®'i=' -ave*'!^* ^ave'’''^ -ave'^^* 


(23) 

(24) 


(25) 


(26) 


P2\/‘2j3, 0 , 0 , flQ, 62 


-ave'i* „*ave*'i^ -ave"!^ 7ave'‘>'‘ -ave"!^ 


where 


7i = 


P2\/2'y3, 0, 0, —/To, 62 


14 do("\/ 2 a) 


(27) 

(28) 

(29) 

(30) 


where Jo(-) and Ji(-) are Bessel functions of the first kind 
and 

Cr® = MV^o) cos(a*®™) cos(d®™) cos(0®™) 

+ Jo(a) sin(Q!*®™) sin(d®™), 


bqvRpi 

_oT2A/d„i , 

72 — 2Jg (V 2a) H--—--, 

bqr Rp2 

Jo(2'/2a) + Jo{2a) — 72 


73 = 


Jo{2y/2a) - 1 


Pi = 2Jo‘^{V2a) — - Jo(2V2a) + Jo{2a) + 1 


P2 = 


V2b [1 - Jo (2^20)] 


4ce Ji(\/2a) 


Po = arccos 


^According to the physical interpretation of the system, here we have 
implicitly restricted E [—7r/2, 7r/2], E [—7r/2, 7r/2], E 

(—tTjTt] to exclude repetitive equilibria. 
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V^Jl{V2a) ^ VcJl{V2a) 

^ qrb‘^B?pl bpi 

62 = -2gr73P2^ - 2qrRp2Jo{V2a). 


The average system (l2^ - (l26l l may converge to one of 
the four equilibria (I27l) - (l30l) under different parameters and 
initial conditions. Each equilibrium corresponds to a torus 
in the vicinity of the source. Around equilibrium dZTl i and 
equilibrium (l28T l. the vehicle converges to a torus with its 
average heading directly towards or away from the source. 
Around equilibrium ( |29] | and equilibrium (l30l l the vehicle 
revolves around the source clockwise or counterclockwise 
and its average heading is more outward than inward. The 
should be real and positive as it represents the average 
distance between the vehicle center and the source. Note that 
Q,*ave^^ave equal to zero for all the equilibria of the 
average system, thus all the tori nearly reduce to annuluses. 
The values of 0^'^^ for different average equilibria are quite 
different, whilts in different motion patterns in the tori. We 
shall formalize these phenomena in the sequel and illustrate 
in the simulations. 

Let and denote the Jacobians of the 

average system (|2^ -(|26]| at equilibria dZTli-dSOll respectively. 
It is obvious that if all the roots of the characteristic equation 
for a Jacobian have negative real parts, the corresponding 
equilibrium is an exponentially stable equilibrium of the aver¬ 
age system. Noting from (iJTT i and (l32l l that the characteristic 
equations of and are the same, we conclude that 
equilibrium (l27l l and equilibrium (|28] | have the same stability 
conditions. This property also holds for equilibrium (l29l l and 
equilibrium (l30l) . Applying Theorem in ll23l . we derive the 
following two propositions. 

Proposition 1: Consider the system (fTTI) -(fT6b with positive 
parameters a, Cq, cq, b, h and 14- Suppose that these pa¬ 
rameters are appropriately chosen so that the Jacobian 
is Hurwitz. For sufficiently large w, if the initial conditions 
rc(0), 0(0), a(0), e(0) are such that the following quantities 
are sufficiently small 


|?'c( 0 ) -r*\- | 7 i||, |a(0)|, |e(0) - qrR^ - ei| 
and 0(0) — arctan ~ f + sg'^(7i) ^ f 


where the sgn(-) is a standard sign function, then the trajectory 
of the vehicle center rc(f) exponentially converges to, and 
remains in the torus 


|a| < 0(l/w), 

|7i| - 0(l/w) < |rc - r*| < |7i| -f 0(l/w). 

Proof: The system (fTTT i- lfThl l is equivalent to the error sys¬ 
tem system dnii-diB, whose solution can be approximated by 
the solution of the corresponding “average system” according 
to the averaging theory in ll23l . If the Jacobian is Hurwitz, 
then both equilibria (l27l l and (l28l l are exponentially stable of 
the average system since the characteristic equations of 
and are the same. By Theorem 10.4 in ll23l . we conclude 
that the error system (fT7] |- (l2Tll has two distinct, exponentially 
stable periodic solutions within 0(l/a;) of the equilibria (|27] | 
and (|28T i. ■ 


Proposition 2: Consider the system (fTTTl -lfThll with positive 
parameters a, Ca, ce, b, h and 14- Suppose that these pa¬ 
rameters are appropriately chosen so that the Jacobian J®'^^ 
is Hurwitz. For sufficiently large uj, if the initial conditions 
rc(0), 0(0), a(0), e(0) are such that the following quantities 
are sufficiently small 


|fc(0) -r*\- P2V^\ ; |a(0)l ) |e(0) - qrR"^ - 62! , 


and either 


or 


0(0) — arctan — p-o 


0(0) — arctan + po 


then the trajectory of the vehicle center rdt) exponentially 
converges to, and remains in the torus 


|a| < 0{l/uj), 

P2\f^z- OiX/uf) < fc - r*\ < p2\/^+ 0 {l/ui). 


Here we do not give the proof of Proposition 2 since it is 
essentially a repetition of the proof of Proposition 1. In both 
propositions, the vehicle center converges to a torus around 
the source. In Proposition 1, the limit of the vehicle’s heading 
is either directly towards or away from the source on average. 
In this case the vehicle stays in the torus as if it comes to a 
full stop, which is quite different from the result of Cochran’s 
work 0. In Proposition 2, the vehicle drifts clockwise or 
counterclockwise in the torus and the limit of its average 
heading is more outward than inward. 

Remark 1: It should be noted that we have assumed the 
distribution of the signal field can be approximated by 

J = f* - qr\rs - r*f 

in the stability analysis. This implies that the proposed con¬ 
trol scheme works for signal fields satisfying the following 
two requirements: (a) the signal strength depends only on 
the distance between the vehicle and the source, i.e., J = 
f{d), d = |rs — r*|; (b) f{d) is strictly concave in the distance 
d. However, the simulation results in Section |V] illustrate that 
the control scheme also works for some other unknown forms 
of the signal fields. 


IV. Parameter Selection 


In this section we further study the stability conditions 
for Proposition 1, and provide a rule on the selection of 
the parameters in Proposition 1. The stability conditions for 
Proposition 2 can be similarly derived. However, their forms 
are fairly complex and lengthy, we only give a brief discussion 
in Appendix 151 

We first note that the Jacobians of the average system (|2^ - 
(l26l l at equilibrium (l27l i and equilibrium ( l28l l are given in the 
following form 



mil 

0 

0 

0 


77151 


0 

TO 22 

TO 32 

0 

0 


0 

W 23 

W 33 

0 

0 


0 

0 

0 

77744 

0 


7771 5 
0 
0 
0 

-h 


(31) 
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U) 


mil 0 0 0 —toi5 

0 m22 -m23 0 0 

0 -m32 TO 33 0 0 

0 0 0 TO44 0 

-TO 51 0 0 0 —h 


(32) 


where 


2V,Jo\V2a) „ 

mil =-- 7;0QrR 

Rpi 2 

mi5 = bJo{V2a), 

m 22 = ^bqrR [3 Jo(2a) - 2] , 


Jo(2'\/2a) + 2Jo(2a) -t- 1 


m23 = bqrR 


Jo (a) 


m32 = 2c, 


2Jo(v^fl) ■ 
— 2bqrRJo{Vba), 
VcMV2a) 


Jo(2v^a) + Jo(2a) + 1 


bpi 


Ji{a), 


/77 T4Jo(-\/2a) ^ ^ /77 s, 

W 33 = -v2ca-r-Ji(V2a), 

bpi 

/- VcMV2a) 
m 44 = — V2ce- 

1 , 


bpi 


-Ji{V2a 


-bqrR 


Jo(2'\/2a) “t" Jo(2a) — 1 


W 51 = 


_2^ KJo(a/ 2«) +2feg,flJo(V2a). 

Oiipi 


Due to the property of determinant ll24l . J®'?^ has the same 
eigenvalues as the following block diagonal matrix 


1 


diag 


W 22 W 23 

TO 32 W 33 


[W 44 ] , 


mil Wi5 
msi —/i 


(33) 


It is clear that the characteristic equation for matrix (l37t is 
the product of the characteristic equations of the three blocks. 
Then we can easily calculate the characteristic equation for 
which is given by 


0 = 


(ujsf - (m22 + m33)ujs + m22m33 - m23m32 
X (tus — m 44 ) 

(ws)^ + {h — mii)ujs — miih — mi^m^i 


(34) 


Note that the characteristic equation for J®'^^ is also equal 
to (I 34 I 1 . thus equilibrium (l27T l and equilibrium (1281) have the 
same stability conditions. 

To guarantee that all the roots of characteristic equation (17^ 
have negative real parts, we invoke the Routh-Hurwitz test ll25l 
and result in the following conditions 


m 22 + m33 

< 

0 , 

(35) 

m22m33 - m23m32 

> 

0 , 

(36) 

m44 

< 

0 , 

(37) 

h — mil 

> 

0 , 

(38) 

—miih — mismsi 

> 

0 . 

(39) 


Substituting the definitions of mu,-- - ,m 5 i into the above 
inequalities, we derive the following stability conditions 


b^qrR4>i < 2vTcaV' /°^^°^ Ji(V2a), (40) 
Pi 

b‘^qrR^2 < 2V2csv /°^^°'h i{y/2a), (41) 


Pi 

214 '^° (^°) < hR+\bqrR^(, 

Pi 2 


JoiV2a) 


Pi 


V2 


4>iJi{V2a) + ^3 Ji(a) 


< 0 , 


4JoW2a ) - <^4 < 0, 


where 


(42) 

(43) 

(44) 


(j)i — 3Jo(2a) — 2, 

4>2 = Jo(2'\/2a) + Jo(2a) — 1, 

1 (^2 + 2) - 4Jo(V5a), 
Jo(v2a) 

(j >4 = Jo(2'\/2a) -f 2Jo(2a) -t- 1. 


Note that inequalities ( |43] ) and (l44l i depend only on the 
parameter a. We restrict the parameter a to the union of two 
intervals and S^, which are defined by = [1.25,1.65] 
and = [1.75,2.5]. The reason is that the constraint 
guarantees the validity of inequalities (i43T l and (i44l) and also 
brings convenience for the design of the other parameters. 
Under this constraint, we have that (pi < 0, (/)2 < 0, pi < 0, 
Ji(-\/2a) > 0, and 


f Jo(-\/2a) >0, if a G S),, 

\ Jo{V2a) <0, if a G S^. 

Now let us consider inequalities (i40l) -(i42li. Observe that the 
right hand side of inequality (i42l i is positive since is positive 
for all positive a. The left hand side of inequality dull is 
negative with a negative pi. Thus inequality (l42l) holds for 
all a G Sj U S^. When a G S^, the inequalities (i40l l and (i4T]) 
hold since their left hand side is negative while their right 
hand side is positive. When a G Sj, we need the following 
conditions 




44 < 


\/2b‘^qrRpipi 
AcaJ(i{\/2a)Ji{\/2a) ’ 
'/2b‘^qrR(p2Pi 


(45) 


(46) 


4ceMV2a)Ji{V2ay 

to satisfy the inequalities (l40l l and dTTl i. 

Summarizing the above, we conclude that the local ex¬ 
ponential convergence to equilibria and (i28l l can be 
guaranteed by selecting appropriate parameters a and I 4 as 
in the following two corollaries: 

Corollary 1: Consider the system (HB-dllll with positive 
parameters a, Cq, cq, b, h and 14- Let the parameter a G 
[1.75,2.5], then for sufficiently large oj, if the initial conditions 
rc(0), P(0), a(0), e(0) are such that the following quantities 
are sufficiently small 


|kc(0) 

and 


- ’’*1 - 7i|, |a(0)l , |e(0) - qrR'^ - ei 
9{0) — arctan > 
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Vehicle Trajectory Projection of vehicle trajectory on x-y plane 



X 



Fig. 4. Simulation results for Corollary 1: (a) Vehicle trajectory; (b) Projection of vehicle trajectory on x — y plane. 


then the trajectory of the vehicle center rc{t) exponentially 
converges to, and remains in the torus 


|a| < 0(l/w), 

7i - 0(l/w) < |rc - r*| < 7i + 0(l/a;). 


Corollary 2: Consider the system (fTTT i- ifThl l with positive 
parameters a, Ca, eg, b, h and 14- Let the parameter a G 
[1.25,1.65] and the parameter 14 be chosen such that Vc < Vc, 
where 


14^ 


\/2b^qrR 


‘iJo{V2a)JiiV2a) 


X min 


f (t’lPi <p2Pi \ 

\ Cq- CQ 


I 


(47) 


then for sufficiently large w, if the initial conditions 
rc(0), 0(0), a(0), e(0) are such that the following quantities 
are sufficiently small 


|re(0) - r*\ + 7iL |a(0)|, |e(0) - qrR'^ - eJ 


and 


0(0) — arctan — tt 


then the trajectory of the vehicle center rdt) exponentially 
converges to, and remains in the torus 


|a| < 0(l/w), 

-7i - 0 { 1 / lu ) < jrc - r*| < -71 + 0 { 1 / lu ). 


The above two corollaries provide a rule on the parameter 
configuration of the control scheme for the static source that 
produces a signal field with spherical level sets. In the two 
corollaries, the vehicle center locally exponentially converges 
to, and remains in the torus represented by equilibrium dZTl) or 
equilibrium (|28]) . Note that the conditions in the corollaries are 
usually stricter than the stability conditions (l40l i- (l44li . In fact, 
we can find many arrays of parameters that satisfy the stability 
conditions (I40l l- (l44l l but violate the conditions in Corollary 
1 and Corollary 2. It is interesting that the biased forward 
velocity I 4 in Corollary 2 must be small enough while it can 
be any positive value in Corollary 1. This distinct feature is 
different from the 2-D case in m, which requires a small 
14 to actuate the vehicle to “stop” near the source. However, 


we still suggest a relatively small I 4 for Corollary 1 since a 
smaller I 4 results in a closer distance to the source. 

Remark 2: In this section, the intervals [1.75, 2.5] and 
[1.25, 1.65] are suggested for the perturbation amplitude since 
they guarantee the validity of inequalities (43) and (44) and 
also bring convenience for the design of the other parameters. 
Besides, a larger perturbation amplitude implies more resistant 
to the noise. In Cochran’s work 0, such a large perturbation 
amplitude would lead to rambling around the source, as a 
constant forward velocity and a large perturbation amplitude 
make the vehicle move fast. When around the source, the 
vehicle moves too fast to estimate the gradient precisely. In 
our control scheme, the forward velocity is designed to be 
positively correlated to the output of the washout filter. Such a 
tunable forward velocity can avoid too fast movements around 
the source. Thus even the perturbation amplitude is chosen 
within [1.75, 2.5] and [1.25, 1.65], the vehicle can estimate 
the gradient correctly and settle down near the source. 

The derivation for stability conditions for Proposition 2 is 
provided in Appendix [B] 

V. Simulation 

Due to different initial conditions, the vehicle may con¬ 
verge to different regions around the source under different 
scenarios. In this section we present simulation results to 
illustrate the behaviors described in Corollary 1, Corollary 2 
and Proposition 2. We also consider locating a source which 
creates a non-quadratic signal field. 

A. Signal fields with spherical and elliptical level sets 

Fig. |4] illustrates the behavior dictated by Corollary 1. The 
map parameters are set as /* = 1, r* = (0,0, 0) and qr = 1, 
and the initial conditions of the vehicle are set as 7’c(0) = 
(1,1,1), a(0) = —7r/2 and 0(0) = —7r/2. The controller 
parameters are chosen as w = 40, a = 2, = 100, eg = 

100, b = 5, h = 10 and 14 = 0.001. Fig. |4] (a) shows the 
trajectory of the vehicle converging to an attractor very close 
to the source. Instead of drifting in a torus like in 0, the 
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0.003/s 
K + H 0.001/s 
-0.001 /s 


Forward velocity of the vehicle 


Vehicle Trajectory 


M 1 ^: ' ' 1 ^ 1 ^ ft ft ft ft ft ft ft ft ft ft ft ft 




\j' \i [j [j' 

399.5s 
Time 

Pitch velocity of the vehicle 


'\J \l I' Ij' I' li I 



Fig. 5. Forwai'd and angular velocities of the vehicle. 


Vehicle Trajectory 




Fig. 7. Vehicle trajectory illustrating Proposition 2. 

Vehicle Trajectory 



Fig. 6. Vehicle trajectory illustrating Corollary 2. 


vehicle slows down nearly to a stop when it is in the torus, 
which is almost in a horizontal plane, as the average azimuthal 
angle between the vehicle and the source is close to zero. 
We can notice more details in Fig. |4| (b), which describes 
the projection of vehicle trajectory on a; — y plane. The limit 
of the average heading of the vehicle is directly towards the 
source, which coincides with the theoretical result in Corollary 
1. Fig. |5] depicts the forward and angular velocities of the 
vehicle in the torus. 

Fig. |6| illustrates the behavior dictated by Corollary 2. 
The map parameters, initial conditions of the vehicle, and 
controller parameters are chosen to be the same as those in 
Fig. m except a = 1.5. As in Fig. @1 the vehicle converges to 
an approximately planar near to the source. In this case the 
vehicle overshoots the source and comes to a “stop”. The limit 
of the average heading of the vehicle is directly away from 
the source, as dictated by the average equilibrium (|28] |. 

Note that the vehicle does not strictly come to a full stop 
in the two simulations though its stop seems evident from 
Fig. m and Fig. |6| In fact, the forward velocity of the vehicle 
oscillates around the biased forward velocity 14 and the yaw 
and pitch velocities oscillate around zero, as shown in Fig. |5] 
When the vehicle settles stably in the torus, it still remains 
moving within a small area. 


Fig. 8. Vehicle locates a source which creates a signal field with elliptical 
level sets. 


Fig. |7] illustrates the behavior dictated by Proposition 2. 
The map parameters, initial conditions of the vehicle, and 
controller parameters are chosen to be the same as those in 
Fig.|6]except 14 = 0.1. The trajectory before entering a torus is 
similar to the one in Fig. [6] However, the vehicle turns around 
and keeps moving with a relatively large forward velocity after 
it overshoots the source. After several times of adjustment of 
direction, it eventually drifts in a planar torus, as shown in 

Fig. Ill 

Our control scheme also allows the vehicle to seek a source 
which creates a signal field with elliptical level sets, as shown 
in Fig. We now assume the nonlinear map takes the form 

f(rs) = 1 - 2a:f - 0.5yf - zf, 

where = [xs, ys,The initial conditions of the vehicle 
and controller parameters are chosen to be the same as those 
in Fig. |4| The vehicle converges to a region near the source 
with its average heading directly towards the source, which is 
similar to the result in Fig. |4| 

B. Non-quadratic and non-concave signal fields 

For some signal fields with non-quadratic maps, our control 
scheme also exhibits abilities to seek the sources, as shown in 
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Vehicle Trajectory 



Fig. 9. Vehicle approaches an acoustic source. 


Vehicle Trajectory 



Fig. 10. Vehicle approaches a source whose signal map is a Rosenbrock 
function. 


Fig. |9] and Fig. [TO] In Fig. |9] we consider an acoustic source 
with unit emitting power. Then, the signal distribution takes 
the form 




1 

47r |rs|^ 


Considering the signal strength is too large near the source, 
we replace J by J', which takes the form 


VI. Conclusion 

We have studied the nonholonomic source seeking problem 
in the 3-D workspace by proposing a new control strategy, 
which extends the work in the 2-D workspace Ool from 
two dimensions to three dimensions, and proved the local 
exponential convergence of the closed-loop system for a static 
source that produces a signal held with spherical level sets. 
Under different parameters the vehicle may virtually “stop” 
or revolve around the source. We also provided a rule on the 
parameter selection for the control scheme. The theoretical 
results were validated through simulations under different 
scenarios. 


Appendix A 

Computation of the Equilibria 

By setting the right hand side of the average error system 
(I22li - (l26l l to be zero, we have 


0 = + be^' 

+ be^ 


0 = 


o = e 


sin 

ave 


Vc)C-‘2bqrRPrC: (48) 

Vc) « 

_^^ave _ 2bqrRCi''^ , (49) 

(50) 


cos 

0 = 2ceqrRPT"^c"‘' 


2bqrR 

cos(a*'^'"®) 




COS sin 
ave 
c 


+ 


(K 


- bqrif^'^f - cossin 

__ i _ Lf ave 

cos(a*®'™) 


0 = -qrif^y - -f 


(51) 

(52) 


We hrst note that when = 0 and = 0, the 

Q, Ct 

equations ( |49] ) and ( fSOl l hold since ^ ave = 0, = 0 and 

sin 

= 0. Then we only need to consider the equations dTsT l. 
dSB and (l5^ . which take the forms 


0 = -r]iJo{V2a) cos{9 ) - ]^hqrRff'^'q 2 , 
0 = —\/2cgqrRf’^’^Ji{V2a) sin(0®'''®) 


+ ^Jo(^a)sin(r™) 


+ ]^bqrR Jo(2v^a)-f Jo(2a) sin(2r™), 
= -qr{f’^’' f + Jo(v^a) cos(0 ), 


(53) 


(54) 

(55) 


J' = - exp(-J) = - exp-^^ 

\ 47r|rs| 

In Fig. [To] we assume the distribution of the signal held is an 
Rosenbrock function, which takes the form 

J = -xl- ivs - xlf -yl- {Zs - ylf. 

The Rosenbrock function is obviously a non-concave function 
and has an isolated maximum at (0,0,0). In both simulations, 
the initial conditions of the vehicle and controller parameters 
are chosen to be the same as those in Fig. ID Simulations show 
the vehicle can well approach the source, as depicted in Fig. |9| 
and Fig. fTOl 



where 


ryi = Uc - bqrif^y - 6e® 


m = 


Joi2V2a) + Jo(2a)j cos(2r™) -f Jo(2a) + 1. 


When sin(0®™) = 0, the equation (l54l l holds and we can 
easily derive equilibria dZTl) and (l28l l. Otherwise we rewrite 
the equations (|53| and equation (l54l) as follows 

Vi _ _ bqrRm 

r”® “ 2Jo(y2a)cos(r'"')’ 

0 = -V2ceqrRrrMV2a) + ^Jo{V2a) 


(56) 


-f bqrR [jo(2^a) -f Jo(2a)\ cos(0‘"™). (57) 
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Substituting ( l56l t into < l57] t, we derive 


fave 
' c 


P2 


(58) 


Combining (l53T l and (fSST l. we figure out fjj™. Then we obtain 
equilibria (|29] ) and (O. 


Appendix B 

Stability Conditions for Proposition 2 
The Jacobians of the average system (l22l i-(l26ll at equilib- 


rium (|29]| and equilibrium (|30]| 

are 






^11 

0 

0 

/l4 

hb 


1 

0 

122 

I 23 

0 

0 


jeq 3 ^ _ 


0 

h2 

I 33 

0 

0 

5 

LU 

^41 

0 

0 

(44 

(45 




_hi 

0 

0 

(54 

-h 



■ 

11 

0 

0 

— ^14 

hb 

1 


0 

I 22 

hs 

0 

0 

jeq 4 : _ 


0 

(32 

I 33 

0 

0 

UJ 

- 

I 41 

0 

0 

(44 

—(45 



51 

0 

0 

—154 


h 


where Uj is the element of the Jacobian of average system 
(Hall-® evaluated at equilibrium (|29] |. We do not give the 
explicit form of lij since it is quite lengthy and easily obtained. 
Both and has the same characteristic equation as the 
block diagonal matrix 


- ■ diag <1 

I 22 hs 
I 32 ^33_ 

UJ 

1 



The characteristic equation is 
0 = [s^ — {I 22 + ( 33 ) 

X + kis^ + I 


/ll 

Zl 4 

hb 

1 

(41 

I 44 

(45 

1 

I 5 I 

hi 

—h 

i 

+ 

to 

to 

h3 — 

(23^32] 


+ fca) , (59) 


where 


ki — h — III — ^44j 

fe = ^11^44 — hlh — Uih — ?45^54 — ^14^41 — ^15^51, 

^3 = hiliih — liiliih + I11I45I54 + I15I44I51 
— I 14 I 45 I 51 — liBhihi- 

According to the Routh-Hurwitz test 1251 . to guarantee that 
all the roots of characteristic equation ( l59b have negative real 
parts, we require the following conditions 


^22 + ^33 < 0) 

(60) 

^33 — I 23 I 32 > 0, 

(61) 

ki > 0, 

(62) 

k2 > 0, 

(63) 

o' 

A 

CO 

(64) 

kik2 — fcg > 0. 

(65) 


The inequalities ( I60H651 are explicit stability conditions for 
Proposition 2, resulting in the following corollary: 


Corollary 3: Consider the system with positive 

parameters a, Ca, eg, b, h and Vc- Let these parame¬ 
ters are appropriately chosen to satisfy inequalities (|60l- 
(l65l . then for sufficiently large uj, if the initial conditions 
rc(0), 0(0), q:( 0), e(0) are such that the following quantities 
are sufficiently small 


|rc( 0 ) -r*\- p 2 V^\ ) |a( 0 )l > |e( 0 ) - QrR'^ - 62 ! 


and either 


or 


0(0) — arctan — /ig 


0(0) — arctan ^ 




Mo 


then the trajectory of the vehicle center rdt) exponentially 
converges to, and remains in the torus 


|a| < 0(l/a;), 

P2'/^ - 0{l/uj) < |rc-r*| < p2\/^+0(l/w)- 
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